91 research outputs found
Taming Reversible Halftoning via Predictive Luminance
Traditional halftoning usually drops colors when dithering images with binary
dots, which makes it difficult to recover the original color information. We
proposed a novel halftoning technique that converts a color image into a binary
halftone with full restorability to its original version. Our novel base
halftoning technique consists of two convolutional neural networks (CNNs) to
produce the reversible halftone patterns, and a noise incentive block (NIB) to
mitigate the flatness degradation issue of CNNs. Furthermore, to tackle the
conflicts between the blue-noise quality and restoration accuracy in our novel
base method, we proposed a predictor-embedded approach to offload predictable
information from the network, which in our case is the luminance information
resembling from the halftone pattern. Such an approach allows the network to
gain more flexibility to produce halftones with better blue-noise quality
without compromising the restoration quality. Detailed studies on the
multiple-stage training method and loss weightings have been conducted. We have
compared our predictor-embedded method and our novel method regarding spectrum
analysis on halftone, halftone accuracy, restoration accuracy, and the data
embedding studies. Our entropy evaluation evidences our halftone contains less
encoding information than our novel base method. The experiments show our
predictor-embedded method gains more flexibility to improve the blue-noise
quality of halftones and maintains a comparable restoration quality with a
higher tolerance for disturbances.Comment: to be published in IEEE Transactions on Visualization and Computer
Graphic
Improved Diffusion-based Image Colorization via Piggybacked Models
Image colorization has been attracting the research interests of the
community for decades. However, existing methods still struggle to provide
satisfactory colorized results given grayscale images due to a lack of
human-like global understanding of colors. Recently, large-scale Text-to-Image
(T2I) models have been exploited to transfer the semantic information from the
text prompts to the image domain, where text provides a global control for
semantic objects in the image. In this work, we introduce a colorization model
piggybacking on the existing powerful T2I diffusion model. Our key idea is to
exploit the color prior knowledge in the pre-trained T2I diffusion model for
realistic and diverse colorization. A diffusion guider is designed to
incorporate the pre-trained weights of the latent diffusion model to output a
latent color prior that conforms to the visual semantics of the grayscale
input. A lightness-aware VQVAE will then generate the colorized result with
pixel-perfect alignment to the given grayscale image. Our model can also
achieve conditional colorization with additional inputs (e.g. user hints and
texts). Extensive experiments show that our method achieves state-of-the-art
performance in terms of perceptual quality.Comment: project page: https://piggyback-color.github.io
- …